Tracking and Digital Documentation of Haptic Manipulation Data using Wearable Sensors

ABSTRACT

A method and system are provided for tracking manipulation data. A wearable sensor array is capable of providing force data, motion data and location data. A body part of a person is covered with the wearable sensor array. The body part of the person is being manipulated through the wearable sensor array, and then a computer system obtains sensor data from the wearable sensor array. The computer system generates a force, motion and location map based on the obtained sensor data which is useful for a variety of purposes as detailed herein.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional PatentApplication 62/892,967 filed Aug. 28, 2019, which is incorporated hereinby reference.

This application is a continuation-in-part of U.S. patent applicationSer. No. 15/683,450 filed Aug. 22, 2017, which is incorporated herein byreference.

U.S. patent application Ser. No. 15/683,450 is a continuation of U.S.patent application Ser. No. 14/270,526 filed May 6, 2014, which isincorporated herein by reference.

U.S. patent application Ser. No. 14/270,526 filed May 6, 2014 is acontinuation-in-part of U.S. patent application Ser. No. 12/371,392filed Feb. 13, 2009, which is incorporated herein by reference.

U.S. patent application Ser. No. 12/371,392 filed Feb. 13, 2009 claimsthe benefit of U.S. Provisional Application 61/029,202 filed Feb. 15,2008, which is incorporated herein by reference.

FIELD OF THE INVENTION

This invention relates to tracking manipulation data using wearablesensors during a medical procedure, a diagnostic or therapy session or amassage.

BACKGROUND OF THE INVENTION

Healthcare providers could benefit from sensors and sensor-enabled toolsthat can be used for capturing, quantifying, characterizing, anddisplaying hands-on maneuvers in a standardized, digital format. Digitaldocumentation of hands-on medical examinations and procedures allows fora standardized method of communication and data sharing both forimproving the understanding of clinical disease and the hands-on skillsnecessary to diagnose and treat disease. Data generated from saidsensors or sensor-enabled tools would be used to indicate specializedroutines, preferences or idiosyncrasies as well as confirmation of theutility of standardized approaches. As digital documentation of hapticor hands-on maneuvers is not in current use in healthcare this techniquecan also be used for clarification and standardization of evidencedbased recommendations which are currently either verbalized or inwritten format.

Similarly, massage therapists, chiropractors or other professionals whouse therapeutic or diagnostic touch could also benefit from sensors andsensor-enabled tools that can be used for capturing, quantifying,characterizing, and displaying hands-on maneuvers in a standardized,digital format. Digital documentation of hands-on therapeutic ordiagnostic examinations and procedures also allows for new communicationmethods and data sharing. As many of the patients seeking these servicesare awake and have their own preferences and personalized outcomes,digital documentation can serve as an important, real-time communicationtool. This also applies to awake patients for medical examinations andprocedures where there is an opportunity to align patient preferenceswith the standard of care.

Digital documentation of hands-on maneuvers can allow for the creationof client preference profiles for future sessions as well asinter-session comparisons or the like. For clinical procedures andexams, the digital approach could lead to auto documentation, comparisonand digital diagnosis using pattern recognition to detect changespatient tolerance and perception as well as differences in practitionerperception and approach. The recorded information would not onlyrepresent practitioner workflow and preferences but would also representorgan mobility and pliability hence a haptic and digital representationof anatomical and pathological variations in human anatomy. For clinicaland therapeutic touch, such technology could lead to the real-timefeedback or sharing on preferred techniques, as well as quantitativefeedback regarding the speed, depth and sequence of tissue or anatomicalmanipulation of various body parts or organs.

No formalized methods or technology tools exist to facilitate digitaldocumentation of haptic manipulations in medicine and surgery. During amedical procedure or examination, a series of technical, cognitive andperceptual events and decisions take place, many of which are notvisible, nor can they be tracked by the human observer. Moreover, thesetechnical, cognitive and perceptual events are difficult to explainverbally. For example, an orthopedic surgeon using his or her hands toassess and reduce a fracture is consciously and subconsciously “feeling”for the tension and relationship between the bones and muscles, whilealso timing their manipulations in such a way to minimize the patient'scounter-response to their planned anatomical manipulation. Only parts ofthis complex interaction between the sense of touch, sight, perceptionand hearing can be seen by an observer. Moreover, a verbal account ofthese sensory inputs, motor outputs and actions are guaranteed to bewoefully inadequate and insufficient in detail for training purposes orclinical documentation. Similarly, a surgeon in the operating roomperforming open, laparoscopic or robotic surgery will have a similar mixof touch, visual and perceptual experiences that define the relationshipbetween the surgeon's hands, instruments and patients' organs. Here too,a verbal account of these sensory inputs, motor outputs and actions areguaranteed to be woefully inadequate and insufficient in detail fortraining purposes or clinical documentation. As such, digitaldocumentation would create a new modality and language for documentinghaptic manipulations. Digital documentation would also enable cross-talkand new communication pathways for healthcare providers and patients

Similarly, when considering diagnostic and therapeutic touch such asmassage or chiropractic treatments, no formalized methods and technologytools exist to facilitate the customer's ability to indicate haptic ortouch preferences during hands-on anatomical manipulations.

There is a need for customer-initiated training and assessment ofsurgical and/or therapy interactions, particularly training to optimizethe sense of touch. In essence, the customer can “train” the therapiston the customer's touch preferences. During a customer-therapistexchange, technical criteria such as “sequence of touch” and “depth” or“completeness of strokes”, traditionally represent techniques that relyprimarily on declarative and procedural knowledge but do not takecustomer preferences into consideration. Other aspects of the touchroutine such as strategic force combinations or serial progression fromone muscle group to another are also important as this providessystematic and therapeutic relaxation while increasing circulation andblood flow. In addition, although these techniques are relativelyuniversal not all of them can be assessed or verbally communicated in anobjective fashion that will routinely yield the perfect customerexperience, as such, a technology tool that enhances communicationcapabilities for haptic interactions is desperately needed.

Anecdotally, some practitioners are reported to simply have “good touch”or “good pair of hands”. Despite the importance of this innate skillwithin hands-on professions, no formalized methods or tools exist toobjectively and quantitatively assess, communicate or define thepreferred range of haptic skills desired of practitioners during thehiring process, initial training, or throughout their professionalcareers. As touch, dexterity and other haptic related psychomotor skillsare extremely difficult to learn by observation alone, methods forproviding digital documentation and communication of these skills arenecessary. Haptic training and testing are the only way to ensure highquality therapeutic and diagnostic interactions and anatomicalmanipulations. Thus, sensors, sensor-enabled tools and visual displaysthat enhance digital documentation and communication of these skills aswell as training of therapists/practitioners and trainees aredesperately needed.

SUMMARY OF THE INVENTION

This invention specifically addresses a need for sensors andsensor-based technology tools to:

1) enable the acquisition of objective data to quantify various touch,medical and surgical procedure techniques,2) support data analytics to characterize, model and compare optimaltechniques (based on expert data) and techniques used by novice orintermediate level practitioners, and3) to allow multi-directional feedback and personal requests includingfrom patients to doctors, customers to therapists and doctor to doctorfor example and4) multi-directional feedback displays can also facilitate relevantcues, guidance and feedback to users for training and assessmentpurposes.

For example, a sensor-enabled tool could collect and analyze data frommultiple experts to develop models of optimal techniques. Datavisualization allows experts to share tips and tricks and facilitateseven greater expertise and significant improvements the quality ofhands-on procedures, exams and therapeutic touch. In addition, noviceand intermediate practitioners and therapists could also use the samesystem to collect their own data and compare it to a database of expertsor highly rated practitioners. Expert data could also be displayed inreal-time as an expert demonstrates a technique, allowing a trainee oranother expert with a different approach to see the data associated withtechnical mastery. In addition, expert data could be displayed to anovice as a form of guidance while the novice attempts to replicate thesame methods and associated forces or motion patterns to achieve aspecific anatomic manipulation. Likewise, therapist's data could bedisplayed to a customer, real time, during an anatomic manipulation suchas a massage, and said customer could use such data to communicatepersonal preferences and “train” the therapist. Data collected in aninitial training or information exchange context could be used by bothinstructors and trainees to accurately assess a novicetherapist/practitioners' skills, identify areas in need of improvement,and provide targeted instruction/feedback to optimize skill acquisitionand customer interactions. The same system could be used bypractitioners throughout their training and career continuums, allowingthem to continually assess and hone their skills over time a part of aself-assessment process.

A benefit of the embodiments of this invention is the use of wearabletechnology to automate clinical documentation in the Electronic HealthRecord which can greatly facilitate documentation accuracy and providemore clinical and anatomical detail than direct human entry via typingor verbal dictation.

In one embodiment, the invention is a method or system for trackingmanipulation data, which includes the steps of:

-   (a) having a wearable sensor array capable of providing force data,    motion data and location data;-   (b) covering a body part of a person with the wearable sensor array;-   (c) manipulating the body part of the person through the wearable    sensor array;-   (d) a computer system obtaining sensor data from the wearable sensor    array; and-   (e) the computer system generating a force, motion and location map    based on the obtained sensor data.

The force, motion and location map could be stored in a database.

Embodiments could further use an imaging or magnetic motion device totrack the hand motion of a person performing the manipulation process.

Embodiments could further include the person receiving the manipulationproviding feedback to the computer system regarding the manipulationprocess and the computer system adding and synchronizing this person'sfeedback to the force, motion and location map in time.

The manipulating step could be a medical exam, a medical procedure, atherapy session or a massage. The manipulating step could be a direct oran indirect manipulation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the method or system for tracking manipulation dataaccording to an embodiment of the invention.

DETAILED DESCRIPTION

FIG. 1 shows the method and system for tracking manipulation data. Awearable sensor array is capable of providing force data, motion dataand location data. A body part of a person is covered with the wearablesensor array. The body part of the person is being manipulated throughthe wearable sensor array, and then a computer system obtains sensordata from the wearable sensor array. The computer system generates aforce, motion and location map based on the obtained sensor data whichis useful for a variety of purposes as detailed herein.

A typical operating room (1 a) is depicted in FIG. 1 and referenced inU.S. Provisional Patent Application 62/892,967 filed Aug. 28, 2019,which is incorporated herein by reference. Expert surgeon (1 b) ismanipulating a laparoscopic instrument while data is being collectedfrom a wearable sensor array on both of her hands (1 c). In addition,her colleague, expert surgeon (1 d), is also wearing said wearablesensors on both of his hands (1 c). As surgeon (1 b) and surgeon (1 d)perform an operation on the patient (1 e), they are able to sharemanipulation data (1 f) relating to their preferred technical approach.Said data could be displayed on an overhead monitor (1 g) or a sterilemonitor in the operative field (1 h). In one embodiment, said data isdisplayed real-time over the operative anatomy of the patient (1 e) oron an operative video (1 i). In another embodiment, said manipulationdata can be displayed independent of operative video (1 i) on a cellphone (1 j) or tablet (1 k). Each user desiring access to themanipulation data (1 f) can decide how to view the data. Viewingmanipulation data in synchronization with surgical video (1 l) enablesanatomical location data to be visualized directly. Without surgicalvideo in the background, anatomical location data (1 m) can be indicatedon the screen. In addition to expert surgeon (1 b) and expert surgeon (1d) sharing operative tips and tricks while donning the wearable sensors,novice surgeon (1 n) can also learn by observing their manipulation datain the operating room via surgical displays (1 g, 1 h) or after theoperation via cellphone (1 j), tablet (1 k) or other display device.

Said wearable sensors are depicted in FIG. 2 and referenced in U.S.Provisional Patent Application 62/892,967 filed Aug. 28, 2019, which isincorporated herein by reference. Said wearable sensors can track motion(2 a), force 2 b, 2 c, 2 d and location 2 a, 2 b, 2 c, 2 d. The motionsensors can be optical or magnetic or any combination of materials thatallow movement tracking. The force sensors can be made of anyengineering or chemical components including ultra-thin, flexibleelectronics mounted onto or mixed within silicon or other materials usedto make artificial skin, adhesives or surgical gloves. Anyone skilled inthe art of wearable sensor technology could define a sensor possible foruse in the manipulation data system. Specific sensor requirements dependon the medical examination or procedure and include individual orcombined sensor arrays capable of providing force, motion or locationdata or a combination of these types of data outputs.

Said arrays should include at least two sensors for each hand. Thesensor could be placed in any location however, certain positions suchas the index finger, thumb or wrist are known to collect the highestquality manipulation data as these locations are the most commonactuators of force and motion. In the case of a single sensor per digitor other hand location, basic hand and finger location information canbe obtained by indicating designated sensor numbers. For example, sensor1 (2 e) could always be used to indicate the thumb location whereassensor 2 (2 f) could always be used to indicate the index fingerlocation. In the case of multiple, visually discrete or microscopicsensors (2 g) per digit (2 b, 2 c) or other hand location (2 d), moredetailed information can be determined regarding what part of the fingeror hand location was used at each moment during an examination orprocedure.

FIG. 3 as referenced in U.S. Provisional Patent Application 62/892,967filed Aug. 28, 2019, which is incorporated herein by reference, depictsan example of less than one minute of surgical manipulation data for theleft (3 a) and right (3 b) hand of a cardiothoracic surgeon. The videoimage in the center shows the surgeon (3 c) with a pair of forceps (3 d)in his left hand (3 e) and a needle driver (3 f) in his right hand (3g). The surgeon is sewing the atrial appendage (3 h) of a bovine heart(3 i) placed in a simulated chest cavity (3 j). The manipulation datafor the left hand (3 a) has a small area with mostly a linear patternindicating that the left hand (3 e) took multiple, small movements fromleft to right and right to left to grasp either the needle from theneedle driver (3 f) or the atrial appendage (3 h) tissue. Themanipulation data for the right hand (3 b) has a larger area. In onepart of the manipulation data, there is a large, repeating loop-like orcircular pattern (3 k) that occurs three times. These three loops arecreated by the motion of the surgeon's right hand (3 g) when pulling thesuture thread through the atrial appendage (3 h) tissue after eachstitch. The surgeon does not put the needle driver (3 f) down or reachfor the mayo stand after placing each stitch. These movements would havegenerated a different pattern in the manipulation motion data. However,during one of the looped motions (3 k) there is a slightly differentpattern noted.

Specifically, in the middle loop, there is a downward dip (31) in theloop pattern near the top of the loop on the left side. This pattern wasgenerated as the surgeon had to move the suture in a slightly differentpattern to get it untangled from the inferior edge of the atrialappendage (3 h).

When using the manipulation data to compare technique, FIG. 4 asreferenced in U.S. Provisional Patent Application 62/892,967 filed Aug.28, 2019, which is incorporated herein by reference, differences can beseen from a variety of perspectives. When comparing the righthandmanipulation data (4 a) of a junior surgeon with the righthandmanipulation data (4 b) of a senior surgeon one can see that the seniorsurgeon has a simpler pattern that takes on the shape of a backwardsletter “C”. While the junior surgeon does have a similar pattern, themotion lines are more diverse and less predictable. Also, the seniorsurgeon spends more time in one area near the bottom of the backwardsletter “C” (4 c).

Additional information can be had when looking at the manipulation dataof the left hands of both surgeons. The left-hand manipulation data (4d) of the senior surgeon appears to have a longer path length comparedto the lefthand manipulation data of the junior surgeon (4 e). Whenconsidering the left- and righthand manipulation data together, itappears that the senior surgeon is more adept or ambidextrous duringthis part of the procedure and uses the left hand (4 d) to assist theright hand (4 b) in conducting a smoother performance compared to thejunior surgeon's right hand (4 a). It is also noted that the seniorsurgeon slows down (dark areas) within area (4 c) more than the juniorsurgeon. In our prior work we have noted that the dark areas are wheremore of the important decisions are being made.

The benefits of storage and access to manipulation data cannot beoverstated. However, in order for manipulation data to be useful, it hasto be intuitive, easy to use and bring value to the current process ofinformation storage and access. Currently, the Gold Standard method fordocumenting the surgical process is verbal dictation. Use of this formatis woefully inadequate in the level of detail and more importantly lacksa basic level of standardization that would allow it to be useful foroutcomes research or empirical investigations of best practices.

FIG. 5, as referenced in U.S. Provisional Patent Application 62/892,967filed Aug. 28, 2019, which is incorporated herein by reference, outlinesa process where sensors at the point of care (5 a) can be used tocapture and digitize the surgical process (5 b). Sensor outputs must beprocessed to improve utility. Data reduction and visualization usingvarious algorithms such as deep learning, image processing, dashboardsand leaderboard display strategies are a critical step (5 c). Theelectronic record is a perfect example where processed sensor data ofsurgical manipulations could benefit the current, text-baseddocumentation (5 d). Automating this process would greatly enhance theclinical workflow by decreasing Electronic Health Record charting timesand improving the quality of that exists in the Electronic HealthRecord.

The concept of tracking and digital documentation of haptic manipulationdata using wearable sensors extends well beyond the operating room.Physical examinations, bedside procedures, radiologic procedures andeven outpatient office procedures and biopsies are all amenable to thisprocess. The clinical breast examination provides a great example, asshown FIG. 6 and referenced in U.S. Provisional Patent Application62/892,967 filed Aug. 28, 2019, which is incorporated herein byreference. In one of our early studies, we built a prototype of awearable bra made of piezoelectric fabric (6 a) that was capable ofgenerating an accurate force profile of the two, previously identifiedstages of the clinical breast examination—“searching” and “palpating” (6b). The force profile for “searching” consists of an average force of15-20 Newtons (6 c) and narrow peak deflections (6 c). The force profilefor “palpating” a mass reaches higher forces (>30 Newtons) in theinitial mass localization period (6 d). In addition, the palpating phaseis characterized by wider peaks (6 e). Breast density and mass locationalso determine the force ranges. The current gold standard fordocumenting clinical breast examination findings is either viadictations, paper documents or computer-based forms (6 f).Unfortunately, these forms represent an over processed version of thepalpatory breast tissue exploration and does not capture certaininformation. During an examination performed by the healthcarepractitioner, it is not uncommon to say the exam was normal as the finalassessment when there may have been two or three areas where more timewas spent due to dense or questionable tissue. Many practitioners do notdocument their full experience. Adding a wearable sensor to this processallows a fast, efficient and automated way of documenting the fullpalpatory experience. A pixel based heatmap of the forces (6 g)generated during the exam could document the examination in more detail.For example, the practitioner may only note that there was abnormalthickening or potentially a mass in region II but may not mention thatthey spent more time and pressed harder in region III and IV beforeconcluding these areas were normal. Another possibility with wearablesensors is the ability to display past examinations in series forcomparison (6 h). In addition, lay persons could be taught theself-examination (6 i) and compare their examination technique andfindings with that of their practitioner (6 j) or a database ofpractitioners (6 k).

The thyroid examination may be conducted using one's hands and or anultrasound. Similar to the breast examination, a wearable sensor cancapture the examination, FIG. 7 as referenced in U.S. Provisional PatentApplication 62/892,967 filed Aug. 28, 2019, which is incorporated hereinby reference. In another pilot study we used a wearable piezo fabricsensor (7 a) to quantify the thyroid examination (7 b). The sensor hadeight total sensing areas including four on the left and four on theright (7 c). The sensors were placed near the hyoid bone (sensor level1), the thyroid cartilage area (sensor level 2), the cricoid andsuperior thyroid gland area (sensor level 3), and the inferior thyroidgland and sternal notch area (sensor level 4). These eight sensorsrevealed for the first time a digital pattern uniquely representative ofa complete thyroid examination (7 b).

Part A represents the point at which the clinician places their hands onthe patient's neck and locates the appropriate anatomy (7 d). Part Brepresents the practitioner asking the patient to swallow as the pressmore firmly on the left side of the neck (7 e). Part C represents thepractitioner asking the patient to swallow as the press more firmly onthe right side of the neck (7 f). This type of data could allow forconfirmation of a complete examination as well as standardization of theforces needed for an accurate, evidence-based examination. Similarly,this same process could be used for ultrasound examination of thethyroid examination.

Clubfoot assessment and casting provides yet another example of aphysical exam and treatment regimen that could benefit greatly fromdigital documentation of procedural manipulation data, FIG. 8 asreferenced in U.S. Provisional Patent Application 62/892,967 filed Aug.28, 2019, which is incorporated herein by reference. The current goldstandard for documenting the physical examination and clinicalassessment of a clubfoot baby is a checklist-based system called thePirani Scoring system (8 a). This system requires several steps, many ofwhich are based on palpation forces such as checking the rigidity ofequinous (8 b), feeling the lateral part of the head of the talus (8 c),and feeling the heel (8 d). Although the scoring system has helped toprovide a structured way of assessing a clubfoot baby, because it islargely based on subjective measures of touch and palpation forces,there may be a wide variation in clinical documentation, treatment andoutcomes for the patients. In our prior work, we have used a variety ofsensors including magnetic motion sensors under physicians' gloves (8 e)and fabric-based sensors sewn directly onto the casting stockinette (8f) to quantify the exact amount of force applied during the clubfootassessment (8 g) and casting process (8 h). As in other examinations atleast two motion sensors are used (one for each hand) and twofabric-based sensors (one over the talus and one near the heel).

Using sensor technology to quantify the manipulation and casting forcesallows for objective and standardized metrics for understanding anddocumenting this childhood disease in the electronic medical record (8i). This process will greatly facilitate clinical research byeliminating the subjective, physician-based documentation that currentlyexists.

In addition to surgical procedures and medical examinations,non-clinical, therapeutic manipulations would also benefit from digitaldocumentation and data capture to facilitate communication, FIG. 9 asreferenced in U.S. Provisional Patent Application 62/892,967 filed Aug.28, 2019, which is incorporated herein by reference. Use of anultrathin, sensor array (9 a) capable of distinguishing anatomicalmassage landmarks would enable a massage therapist (9 b) to adjust thedepth, frequency and direction of their hands-on manipulation of variousbody parts based on verbal feedback or confirmation by the customer (9c). In addition, another option for improving communication duringmassage is for the customer (9 c) to point to a screen (9 d) indicatingthe preferred areas for specific manipulation. The customers screen (9d) could be shared with video or VR glasses (9 e) or a larger roomscreen (9 f). In addition, the wireless sensor (9 a) could communicatedirectly with the VR glasses (9 e) or screens (9 d,9 f). As both massagetables (9 g) and massage chairs (9 h) both have face holes (9 i), thistype of communication is greatly facilitated by customer visualizationof massage palpation data on a screen (9 d, 9 f). For example, the red(9 j) and yellow (9 k) areas on the screens (9 d, 9 f) could indicatethe areas the client prefers the deepest massage. Digitally recordedsessions and aggregated client profiles could be used to provideguidance to massage therapists just prior to and during subsequentsessions with the same client.

FIG. 10 as referenced in U.S. Provisional Patent Application 62/892,967filed Aug. 28, 2019, which is incorporated herein by reference,summarizes the method for sensor-enabled massage communication anddigital documentation. The client (10 a) may input directly into acomputer (10 b) or database that feeds into a practitioner or massagetherapist display (10 c) or back to the client through a client display(10 d). Likewise, a sensor overlay (10 e) can store real-time orhistorical information on a computer (10 b) that the client orpractitioner may respond to through each of their respective displays(10 d, 10 c).

FIG. 11 as referenced in U.S. Provisional Patent Application 62/892,967filed Aug. 28, 2019, which is incorporated herein by reference,summarizes the method for sensor-enabled medical communication anddigital documentation. The surgeon or medical practitioner (11 a) exertsmovements and forces during the course of anatomical manipulations (11b) either with their hands or surgical instruments and generatesmanipulation data (11 c). Said manipulation data (11 c) can either bedisplayed real-time (11 d) to facilitate communication or training withother medical practitioners (11 a) or stored in a database (11 e) ormedical record for future communication or training use or clinicalresearch. Both physician factors (11 f) and patient factors (11 g)independently effect the manipulation data (11 c). For example, aninexperienced practitioner (11 a) may yield manipulation data (11 c)that represents specific MD factors (11 f) in the manipulation data (11c) that may indicate specific technical deficiencies for example.Likewise, patient factors (11 g) may demand specific surgicalmanipulations. In this instance, the manipulation data (11 c) may helpto characterize what it takes to alleviate specific disease processes.For example, in a patient with dense adhesions the surgeon may need touse multiple, short bursts of movement with high frequency but lowvelocity hence, the medical practitioner (11 a) will generatemanipulation data (11 c) because of the dense adhesions, a specificpatient factor (11 g).

The manipulation data has additional benefits beyond digitaldocumentation and communication. FIG. 12 as referenced in U.S.Provisional Patent Application 62/892,967 filed Aug. 28, 2019, which isincorporated herein by reference, shows how digital manipulation data(12 a) can be used to expedite the review of video for personal learningor coaching. A major abdominal operation is used as an example. Thisoperation yielded four hours of operative video and digital manipulationdata (12 a). Instead of looking at four hours of video, specific areasof the motion/manipulation data (12 a) can be targeted. For example, thefirst target chosen is an unexpected segment of motion (12 b). Becausethe motion data is synchronized with the video data, if one clicks orhovers over the targeted motion area (12 b), then they will be directedto video segment 15 (12 c) which shows the surgeon in the process ofcauterizing some recently ligated tissue. The second target chosen is asegment of motion (12 d) that is expected for this procedure. Hoveringover this motion segment (12 d) will reveal video segment 47 (12 e)which shows routine dissection. Other data types could also besynchronized with digital manipulation data including audio data (12 f),cognitive data (12 g) and practitioner physiologic data (12 h). Anycombination of these data streams can be used to identify surgeon orpractitioner factors that effects patient outcomes. Our prior work hasshown that surgeons who have an accurate and efficient surgical protocolwith excellent outcomes have similar motion patterns that generated thatsame digital manipulation data (12 a). In addition, these surgeonsdemonstrate significantly higher use of team language in their audiodata (12 f); significantly higher use of executive function and lowercognitive load in their cognitive data (12 g) as well as low physiologicparameters (12 h).

What is claimed is:
 1. A method of tracking manipulation data,comprising: (a) having a wearable sensor array capable of providingforce data, motion data and location data; (b) covering a body part of aperson with the wearable sensor array; (c) manipulating the body part ofthe person through the wearable sensor array; (d) a computer systemobtaining sensor data from the wearable sensor array; and (e) thecomputer system generating a force, motion and location map based on theobtained sensor data.
 2. The method as set forth in claim 1, wherein theforce, motion and location map are stored in a database.
 3. The methodas set forth in claim 1, further comprising using an imaging or magneticmotion device to track the hand motion of a person performing themanipulation process.
 4. The method as set forth in claim 1, the personreceiving the manipulation providing feedback to the computer systemregarding the manipulation process and the computer system adding andsynchronizing this person's feedback to the force, motion and locationmap in time.
 5. The method as set forth in claim 1, wherein themanipulating step is a medical exam, a medical procedure, a therapysession or a massage.
 6. The method as set forth in claim 1, wherein themanipulating step is a direct or an indirect manipulation.